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Abstract — We consider the design of cognitive Medium Access Control (MAC) protocols enabling a secondary (unlicensed) transmitter- 
receiver pair to communicate over the idle periods of a set of primary (licensed) channels. More specifically, we propose cognitive 
MAC protocols optimized for both slotted and un-slotted primary networks. For the slotted structure, the objective is to maximize the 
secondary throughput while maintaining synchronization between the secondary pair and not causing interference to the primary 
network. Our investigations differentiate between two sensing scenarios. In the first, the secondary transmitter is capable of sensing 
all the primary channels, whereas it senses only a subset of the primary channels in the second scenario. In both cases, we propose 
blind MAC protocols that efficiently learn the statistics of the primary traffic on-line and asymptotically achieve the throughput obtained 
when prior knowledge of primary traffic statistics is available. For the un-slotted structure, the objective is to maximize the secondary 
throughput while satisfying an interference constraint on the primary network. Sensing-dependent periods are optimized for each 
primary channel yielding a MAC protocol which outperforms previously proposed techniques that rely on a single sensing period 
optimization. 
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1 Introduction 

The radio spectrum resource is of fundamental importance 
to wireless communication. Recent reports show that most 
available spectrum has been allocated. However, most of li- 
censed spectrum resources are under-utilized. This observation 
has encouraged the emergence of dynamic and opportunis- 
tic spectrum access concepts, where secondary (unlicensed) 
users (SU) equipped with cognitive radios are allowed to 
opportunistically access the spectrum as long as they do not 
interfere with primary (licensed) users (PU). To achieve this 
goal, the secondary users must monitor the primary traffic in 
order to identify spectrum holes or opportunities which can be 
exploited to transfer data [1]. 

The main goal of a cognitive MAC protocol is to sense the 
radio spectrum, detect the occupancy state of different pri- 
mary channels, and then opportunistically communicate over 
unused channels (spectrum holes). Specifically, the cognitive 
MAC protocol should continuously make efficient decisions 
on which channels to sense and access in order to obtain 
the most benefit from the available spectrum opportunities. 
Previous work on the design of cognitive MAC protocols has 
considered two distinct scenarios. In the first, the primary 
network is slotted (e.g., [2], [4], [5], [7], [8] and [13]) whereas 
a continuous structure (un-slotted) of the primary channels is 



O. Mehanna and A. Sultan are with the Wireless Intelligent Networks 
Center (WINC), Nile University, Cairo, Egypt. 
E-mail: omar. mehanna® nileu. edit. eg, asultan@nileuniversity. edu. eg 
H. El Gamal is with the Department of Electrical and Computer Engineer- 
ing, Ohio State University, Columbus, USA. 
E-mail: hel gamal® ece.osu.edu 
This work was presented in part at ICC '09. 

This work was partially supported by the Egyptian NTRA and the National 
Science Foundation (NSF). 



adopted in the second set of works (e.g., [3], [10], [11] and 
[14]). In this work we propose decentralized cognitive MAC 
protocols for each of the two models. 

For the slotted structure, two cases are considered. The 
first assumes that the secondary transmitter can sense all the 
available primary channels before making the decision on 
which one to access. The secondary receiver, however, does 
not participate in the sensing process and waits to decode 
on only one channel. This is the model adopted in [4] and 
is intended to limit the decoding complexity needed by the 
secondary receiver. In the sequel, we propose an efficient 
algorithm that optimizes the on-line learning capabilities of 
the secondary transmitter and ensures perfect synchronization 
between the secondary pair. The proposed protocol does not 
assume a separate control channel, and hence, piggybacks the 
synchronization information on the same data packet. 

In the second scenario, the secondary transmitter can only 
sense a subset of the available primary channels at the begin- 
ning of each time slot. This model was studied in [5] where the 
optimal algorithm was obtained by formulating the problem as 
a Partially Observable Markov Decision Process (POMDP). 
Unfortunately, finding a computationally efficient version of 
the optimal solution for this problem remains an elusive task. 
By re-casting the problem as a restless multi-armed bandit 
problem, a near-optimal index policy was proposed in [7]. 
The authors of [5] and [7], however, assumed that the primary 
traffic statistics (i.e., Markov chain transition probabilities) 
were available a-priori to the secondary users. Here, we 
develop blind MAC protocols where the protocol must learn 
the transition probabilities on-line. This can be viewed as the 
Whittle index strategy of [7] augmented with a similar learning 
phase to the one proposed in [8] for the multi-armed bandit 
scenario. Our numerical results show that the performance of 
this protocol converges to that of the Whittle index strategy 
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with known transition probabilities [7]. 

Under the un-slotted primary network set-up, we first as- 
sume that the SU radio can be tuned to any combination of 
the primary channels at the same time. This can be achieved 
by an Orthogonal Frequency Division Multiplexing (OFDM) 
technique with adaptive and selective allocation of OFDM 
subcarriers to utilize any subset of licensed channels at the 
same time. The SU aims at maximizing its throughput (i.e., 
maximizing the opportunities discovered and accessed in all 
primary channels) while imposing minimal interference to the 
primary network. A similar model was adopted in [3], where 
the authors developed an optimal sensing period for each 
of the primary channels by optimizing the tradeoff between 
the sensing overhead resulting from frequent sensing of the 
channels and the missed opportunities in the primary channels 
due to infrequent sensing. However, it was assumed that if a 
primary transmission is resumed on a channel, the SU will 
discover this return, via the help of a Genie, and immediately 
evacuate the channel, thereby causing no interference to the 
primary transmissions. In this work, we relax this Genie- 
aided assumption and impose an interference/outage constraint 
on each primary channel. In [11], an optimal sensing period 
satisfying a primary network interference constraint was de- 
veloped. However, the approximations made in [11] deviate 
considerably from the true values. More importantly, we show 
that by introducing two different sensing periods, a period if 
the channel is sensed free and a different period if the channel 
is sensed busy, the performance can be substantially improved. 
In particular, this performance improvement becomes more 
significant when there is a large difference between the ex- 
pected time a primary channel is busy and the expected time 
it remains idle. Finally, we consider the scenario when the SU 
radio can be tuned to only one channel. A SU in this case 
shall try to access a primary channel as long as it is free. 
When this channel switches to busy, the SU shall search other 
primary channels until a free channel is identified. A similar 
model was adopted in [10], where an optimal sequence of 
primary channels to be sensed was proposed. This optimal 
sequence aimed at minimizing the average delay in finding 
a free channel. Here, we extend this work by finding the 
period a free channel shall be accessed in order satisfy an 
interference/outage constraint on the primary network, which 
was not considered in [10]. 

The rest of the paper is organized as follows. Section |2] 
presents our modeling assumptions. The proposed cognitive 
MAC protocols for slotted primary networks are developed 
in Section [3] whereas the un-slotted scenario is investigated 
in Section [4] Numerical results for our proposed strategies 
are reported in Section [5] Finally, Section [6] summarizes our 
conclusions. 

2 Network Model 

2.1 Primary Network 

We consider a primary network consisting of TV indepen- 
dent channels. The presence or absence of primary users in 
each channel can be modeled as alternating time intervals 
of busy and free states with random durations. For channel 
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Fig. 1 . The Gilber-Elliot channel model. 



i G 1, 2, . . . , TV, we model the sojourn time of a busy period 
as a random variable Tf with the probability density function 
(p.d.f.) fT°(y),y > 0. Similarly, the p.d.f. of the sojourn 
time in a free period is given as f T i(x),x > 0. Busy and 
free periods are assumed to be independent and identically 
distributed (i.i.d.). We also assume that busy and free periods 
are independent of each other. The state of channel i at time 
t, Si(t), is equal to 1 if the channel is free and if busy. 

2.2 Secondary Pair 

The SU can sense any of the primary channels in order to 
identify the presence of a PU. After sensing the channel, the 
SU applies the channel access strategies as described in Sec- 
tions [3] and |4] The performance of the sensing stage is limited 
by two types of errors. If the secondary transmitter decides 
that a free channel is busy, it will refrain from transmitting, 
and a spectrum opportunity is overlooked. This is the false 
alarm situation, which is characterized by the probability of 
false alarm Pfa- On the other hand, if the detector fails 
to classify a busy channel as busy, a miss detection occurs 
resulting in interference with primary user. The probability of 
miss-detection is denoted by Pmd- If energy detection is used 
as a sensing method [12], the minimum required sensing time 
T s that satisfies a certain desired Pfa an d Pmd is given by: 

T s = T [Q^(Pfa) - Q-\l - Pmd)V\ + 2^\ 2 ^ 2 (1) 

Is 

Where, f s is the sampling frequency, Q(x) is the tail proba- 
bility of a zero-mean unit-variance Gaussian random variable 
and a is the PU signal-to-noise ratio [12]. 

3 Slotted Primary Network 

In this section, we consider the case of discrete probability 
distributions for the free and busy periods. We model the 
duration of these intervals (in terms of the number of time slots 
they occupy) as geometrically distributed random variables. 
From the memoryless property of the geometric distribution, 
we can model the primary users' traffic in each channel 
by the two state Markov chain depicted in Figure Q] The 
channel state transition matrix of the Markov chain is given 

" pOO pOl " 

plO pll 
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for a block of T time slots. We use j to refer to the time-slot 



by P l = 



We assume that P; remains fixed 



index j G {1, • ■ ■ ,T}. 
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We assume that the secondary transmitter can sense L 
channels (L < N) and can transmit on only one channel 
in each slot if the channel it chooses to access is sensed to 
be free. In this section, Si(j) denotes the state of channel 
i at time slot j as sensed by the transmitter, which might 
not be the actual channel state Si(j). The secondary receiver 
does not participate in channel sensing and is assumed to 
be capable of accessing only one channel [4]. This assump- 
tion is intended to limit the decoding complexity needed by 
the secondary receiver. Another motivation behind restricting 
channel sensing to the transmitter is the potentially different 
sensing outcomes at the secondary transmitter and receiver 
due to the spatial diversity of the primary traffic which can 
lead to the breakdown of the secondary transmitter-receiver 
synchronization. Overall, successful communication between 
the secondary transmitter and receiver occurs only when: 1) 
they both decide to access the same channel, and 2) the 
channel is sensed to be free and is actually free from primary 
transmissions. 

Our proposed cognitive MAC protocol can be decomposed 
into the following stages: 

> Decision stage: The secondary transmitter decides which 

L channels to sense. Also, both transmitter and receiver 

decide which channel to access. 

• Sensing stage: The transmitter senses the L selected 
primary channels. 

• Learning stage: Based on the sensing results from the 
sensing stage, the transmitter updates the estimated pri- 
mary channels' probability transition matrix Pi. 

• Access stage: If the access channel is sensed to be free, 
a data packet is transmitted to the secondary receiver. 
This packet contains the information needed to sustain 
synchronization between secondary terminals, and hence, 
synchronization does not require a dedicated control 
channel. The length of the packet is assumed to be large 
enough such that the loss of throughput resulting from 
the synchronization overhead is marginal. 

• ACK stage: The receiver sends an ACK to the transmitter 
upon successful reception of sent data. 

3.1 Full Sensing Capability: L = N 

In this subsection we assume that the secondary transmitter 
can sense all N primary channels at the beginning of each 
time slot. Since the receiver doesn't participate in sensing, 
and in order to sustain the transmitter-receiver synchronization, 
the secondary pair must share the same variables which are 
used to decide upon the channel to be accessed. We refer to 
f2(j) = [uii(j),--- ,wat(j)] as the belief vector at slot j, where 
uJi(j) is the probability that Si(j) = 1. Given the sensing 
outcomes in slot j, the belief state in slot j + 1 can be obtained 
recursively as follows: 

(Poi if Si(j)=0 

<*(j'+l)=<i£ if Si(j) = l 

\wi(j)Pli + (1 — w i(j))-foi if * n °t sense d at j 

(2) 

Due to the different sensing roles between the secondary 
transmitter and receiver, we introduce the vector Ct(j) as 



the common or shared belief vector between the secondary 
transmitter and receiver. The initial packet sent to the receiver 
includes estimates for the transition probabilities, and £1(1). 
Once the initial communication is established, the secondary 
transmitter and receiver implement the same spectrum access 
strategy described below for j > 1. 

1) Decision: At the beginning of time slot j, and using be- 
lief vector the secondary transmitter and receiver 
decide to access channel: 



jnax^ [uji(J)Bi 



(3) 



2) Sensing: The secondary transmitter senses all chan- 
nels and captures the sensing vector = 
[Si(j), ■ ■ ■ ,S N (j)], where_Si(j) = 1 if the zth channel 
is sensed to be free, and Si(j) = if it is found busy. 
Si(j) might be different than Si(J) due to sensing errors. 
Note that the decision stage precedes the sensing stage 
in order to maintain the synchronization between the 
secondary terminals. 

3) Learning: Based on the sensing results, the transmitter 
updates the estimates Pf 1 and P^ 1 for all primary 
channels as explained below. 

4) Access: If Si*(j) = 1, the transmitter sends its data 
packet to the receiver. The packet includes P 4 01 
and Pi 1 . In addition, if the transmission at slot j — 1 
has failed, the transmitter sends f2(j), which is the belief 
vector computed at the transmitter based on its observa- 
tions. If the receiver successfully receives the packet, it 
sends an ACK back to the transmitter. Parameter Ki- (j) 
is equal to unity if an ACK is received by the transmitter, 
and zero otherwise. If the channel is free, the forward 
transmission and the feedback channel are assumed to 
be error- free. 

5) Finally, the transmitter and receiver update the common 
belief vector f2(j + 1) such that: 

If K t *[j) = 1: 



[PI 1 

u>i(j+l) = I API 1 
If Ki.(j) = 0: 



(l-^)A 01 



if i = i*(j) 

if i^i*(j),S i (j) = l 

(4) 



where: 



= J A A 11 + (l - A) P? 1 if i = 



A = Pr(Si(j) = l\Si(j) = 1) 



Ci = Pr(Si(j) = l\Si(j) = 0) 



(1 - P FA )u>i(j) 



(1 - Pfa)^U) + Pmd(1 - u>i(j^ 



PFA^i(j) 



PFA&iU) + (1 " Pmd)(1 ~ 

(7) 



Di = Pr(Si* (j) = l\Ki»(j) = 0) 



PFA^iU) 



PFA^iii) + (i -^i(i)) 



(8) 
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Pf 1 and P/ 1 are the most recent shared estimates of ith 
channel transition probabilities. Obviously, in case of perfect 
sensing, A, = 1, C 1 , — and A = 0. 

In addition, the transmitter computes another belief vector, 
0(j + 1), based on its observations: 

If Ki.{j) = 1, cj i (j + l)=Q i (j + l) 

If AV(j)=0: 

f M" + (1 - A) P? 1 if * * i*(j),Si(j) = 1 
^(i+1) = ^ QP/ 1 + (1 - Ci) P 4 01 if * ^ i*(j),Si(j) = 

[A-P/ 1 + (1 - A)-?? 1 if i = i*(j) 

(9) 

where Aj, Ci, and A are the same as A,, Ci and A with 
o)i(j) replaced by Note that = and 0(j + l) 

differs from Ct(j + 1) only when AV (j) = 0. If transmission 
succeeds at the jth time slot after one or more failures, the 
transmitter and receiver set = 0(j) before computing 

n(j + 1). 

It is noted that although f2j(_j) is the updated belief vector 
which is available to the transmitter, the transmitter and 
receiver use the degraded f2(j) in the decision stage instead, 
in order to maintain the synchronization. So, as an analytical 
benchmark, we have the following upper-bound on the achiev- 
able throughput in this scenario. Assuming that the delayed 

( 7 — 1 ) 

side information of all the primary channels' states 5, is 
known to the secondary receiver as well as the transmitter, the 
expected throughput per slot is given by: 



«= E -EE 

Sjv=0 S 2 =0Si=0 



n 



P Sl ) (max [P Si iBi. 



(10) 



where Ps t i denotes the state transition probability for channel 
i from state Si = (0, 1) to the free state, and Ps ; is the Markov 
steady state probability of channel i being free or busy. The 
first term in the summation corresponds to the probability that 
the N channels are in one of the 2 N states, and the second term 
represents the highest expected throughput given the current 
joint state for the N channels. The loss in throughput resulting 
from the use of f2(j) instead of in the decision is 

illustrated in Figure [5] 

Since we assume that traffic statistics on primary channels 
(Pi) are unknown to the secondary user a-priori, the secondary 
user needs to estimate these probabilities. When continuous 
observations of each channel are available, each channel can 
be modeled as a hidden Markov model (HMM). An optimal 
learning algorithm for HMM is described in [6] using which 
the transition probabilities, Pfa, and Pmd can be estimated. 
However, we here adopt a simple Bayesian learning method. 
Assume that Pf 1 and P/ 1 are random variables with dis- 
tributions / t 01 (x) and f^iy) defined on [0,1]; respectively. 
After sensing all the primary channels at the beginning of each 
time slot, and depending on the previous state of the channel, 
the posterior distribution of Pj 01 can be updated according to 
Bayes' rule; i.e., 



xff{x) 



/°ii(*)|{oi},- 



f^ l {x)\m j = 



(11) 



(12) 



where the event {01}j represents the state transition from 
busy at time j — 1 to free at time j. Also, the event {00}^ 
represents the state transition from busy at time j — 1 to busy 
at time j. The posterior distribution of P/ 1 can be updated 
similarly. In addition, after sensing all the primary channels 
at the beginning of each time slot, the secondary transmitter 
shall keep track of the following metrics for each channel: 

• Number of state transitions from busy to busy: 

N?°(j) = Y^(l-Si(l))(l-Si(l + l)) d3) 

• Number of state transitions from busy to free: 

j-l 

N? 1 U)=^2(l-S i U))S i (l + i) (14) 
1=1 



• Number of state transitions from free to busy: 



N?°(j) 



j'-i 

E 

i=i 



Si(l)(l-Si(l + 1)) 



• Number of state transitions from free to free: 



E 

i=i 



Si{l)S t (l + l) 



(15) 



(16) 



Thus, if we assume that at j = 0, P^ 1 is uniformly 
distributed in [0,1] (i.e., /^(x) = 1), or in other words no 
prior information about Pf 1 is available, then using equation 
(TlTT i it can be shown that fj X (x) satisfies the following Beta 
distribution: 



f°\ x \N™(j) = N™;N^(j) = N 01 ) = 



(N°°+N 01 + l)\ N oi 

7V01! + jv00| x 



(1-x) 



N 



(17) 



Finally, the expected value of /^(x), obtained from equa- 
tion (JTTJ, gives the following best estimate for P" 1 at time 



pru)= xf^( X )dx = 



(18) 



Using the same approach, the best estimate for P/ 1 at time 
j is given by: 



(19) 



This learning strategy can be easily applied to the situation 
where the primary traffic statistics (Pj) changes with time. One 
simple idea is to consider only a fixed number of previous 
sensing samples in estimating Pj (i.e., using a sliding window 
of samples for the estimation) 
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In order to share the channel transition probabilities between 
the secondary transmitter and receiver, as dictated by the pro- 
posed strategy, any updates in the values of N®°(j), N^°{j), 
N^ 1 (j) and A^ 01 (j) must be sent within the transmitted packet. 
If Ki-(j) = 1, the transmitter and receiver update Pj 01 (j) 
and Pl l {j). Otherwise, the transmitter only updates N®°(j) , 
Nl°(j), Nl 1 ^) and N? 1 ^), but uses the old values, available 
at the last successful transmission, in the decision phase. 

In a nutshell, the proposed algorithm uses the full sensing 
capability of the secondary transmitter to decouple the ex- 
ploration (i.e., learning) task from the exploitation task. After 
an ACK is received, both nodes use the common observation- 
based belief vector to make the optimal access decision. On the 
other hand, in the absence of the ACK, both nodes cannot use 
the optimal belief vector in order to maintain synchronization. 
In this case, the proposed algorithm opts for a greedy strategy 
in order to minimize the time between the two successive 
ACKs. 

A final remark is now in order. Assuming that P/ 1 = PP 1 , 
the probability of channel i being free, Ps i= i, becomes 
independent of the previous state, i.e., Ps i= i = Pi 1 = P® 1 . 
In this case, the optimal strategy, assuming that the transition 
probabilities are known, is for the secondary transmitter to ac- 
cess the channel i* = arg max [Pg i= iPi] and the expected 

i=l,"- ,N 1 

throughput becomes: R = max [Ps i= iBi] [8]. Assuming, 

i=l,"' ,N 1 

however, that the transition probabilities are unknown but 
both nodes know that P/ 1 = Pf 1 , one can estimate each 
channel free probability as Ps i= i = Nl(j)/j, where N}(j) 
is the number of times channel i was sensed to be free until 
time slot j. In Section [5] we quantify the value of this side 
information by comparing the performance of this strategy 
with our proposed universal algorithm that does not make any 
prior assumption about the transition probabilities. 



3.2 The Restless Bandit Scenario: L < N 

Here, we assume that the secondary transmitter can sense only 
a subset L < N of the primary channels at the beginning of 
each time slot. Obviously, the problem here is not as simple as 
the L = N case since the secondary transmitter has to decide 
on which L channels to sense at each time slot in addition to 
the channel the transmitter and receiver decide to access. This 
opportunistic spectrum access network can be modeled as a 
partially observable Markov decision process (POMDP) where 
the channel sensing and access of a MAC protocol correspond 
to a policy for this POMDP [5]. The design objective is to 
determine, in each slot, which channel to sense so that the 

T 

£ 



Riti) 



obtained in T slots 



expected total reward: E 

is maximized, where Ri(j) = Si(j)P>i. It has been shown 
that for any j, the belief vector 0(j) = [u>i(j), ■ ■ ■ , conU)] 
is a sufficient statistic for the design of the optimal action 
in slot j [16]. A policy for a POMDP is thus given by a 
sequence of functions, each mapping from the current belief 
vector to the sensing and access action to be taken in slot 
j. Unfortunately, finding the optimal policy for a general 
POMDP is computationally prohibitive. In [5], the authors 



proposed a reduced complexity strategy based on the greedy 
approach that maximizes the per-slot throughput based on 
already known information (i.e., at time slot j, transmit on 
channel = arg max [uii(j)Bi]). In a more recent work 

i=l,--- ,N 

[7], the problem was re-casted as a restless bandit problem. 

Restless Multi-armed Bandit Processes (RMBP) are gen- 
eralizations of the classical Multi-armed Bandit Processes 
(MBP). In the MBP, a player, with full knowledge of the 
current state of each arm, chooses one out of N arms to 
activate at each time and receives a reward determined by the 
state of the activated arm. Only the activated arm changes its 
state while the states of passive arms are frozen. The objective 
is to maximize the long-run reward over the infinite horizon 
by choosing which arm to activate at each time. The solution 
to the multi-armed bandit problem should be able to maintain 
a balance between the exploration and exploitation in order to 
maximize the total reward. Whittle [9] introduced the RMBP 
which allow multiple arms to be activated simultaneously and 
passive arms to also change states. In each slot, the user 
chooses one of two possible actions to make a particular arm 
passive or active. Whittle's index measures how attractive it 
is to activate an arm based on the concept of subsidy for 
passivity. In other words, Whittle's index for a channel is the 
minimum subsidy that is needed to move a state from the 
active set to the passive set. In [7], the Whittle index Wi(j) 
was obtained in closed form and was used to construct a more 
efficient medium access policy than the greedy approach. The 
Wi(j) given in [7] can be viewed as a combination of the 
immediate reward represented by u>i(j) and a learning reward 
obtained from observing the state of the channel. Based on 
this Whittle index formulation, the maximum reward obtained 

N 

at each time slot is given by R(j) = Ri(j), where: 

i=i 



W l {j) if <•;./; 

Uj) = \ w t {j) - wi(j) if i G U L -i(j) 
otherwise 



arg max [W t {j)Bi 

i=l, ••• ,N 



(20) 



and Ul—\{j) represents the set of L — 1 channels with the 
largest L — 1 values of (W 4 (j) — u>i(j)) not including channel 
= arg max [Wi(j)Bi\. Knowing the states of the set 

i=l,-" ,N 

of channels J7l_i(j) gives the largest observation reward (i.e., 
exploration) which enhances future access decisions. 

Here, we relax the assumption of the a-priori available tran- 
sition probabilities at the secondary transmitter/receiver. This 
adds another interesting dimension to the problem since the 
blind cognitive MAC protocol must now learn this statistical 
information on-line in order to make the appropriate access 
decisions. Inspired by the previous results of Lai et al. in 
the multi-armed bandit setup [8], we propose the following 
simple strategy. The N primary channels are divided into [~^] 
channel groups. Then, at the beginning of the T slots, each 
of the of the channel groups are continuously monitored for 
an initial learning period (LP) to get an estimate for P* 1 and 
P? 1 . In summary, the proposed strategy works as follows: 

1) Initial learning period: Each group of channels are 
continuously sensed for LP time slots. At the end 
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of the learning period, the transition probabilities are 

701 _ N^(j)+i An, 



estimated as P" (j) 



2) Decision: At the beginning of any time slot (j > |~^] x 
LP), the secondary transmitter and receiver decide to 
access channel = arg max [Wi(j)Bi\. 

i=l,--- ,N 

3) Sensing: The secondary transmitter senses the L — 1 
channels in the set J7i_i(j) in addition to channel 

The sensing vector for the selected L channels is 

captured 

4) Learning: if = - 1), update TV/ 1 , Nl , N? 1 , 
N?°, Pi 1 , and Pf. 

5) Access: If Si*(j) = 1, the transmitter sends its data 
packet to the receiver. The packet includes Pj 01 
and P 11 , In addition, if the transmission at slot j — 1 
has failed, the transmitter sends 

6) The transmitter and receiver calculate Cl, while the 
transmitter calculates fi : 

UKpV) = 1: 



A 11 

W + (1 - Ai) poi 
C t P^ + (1 - Ci) Pf 

OiO'^ + a-OiCj))^ 1 



if % = 

ifi^C/i_i(i) 

(21) 



+ 1) = + 1) 



If Ki.{j) = Q: 



Wi(j+1) 



Ap/ 1 + (i - A) Pf 1 if i = 

OiO')^ 1 + (1 " OiCl))^? 1 ^ * ^ **(*') 



(22) 



'^p/ 1 + (i - A^Pf 1 if i e u L -i(j),Si(j) = i 
c^ 11 + (l - Cj) pf 1 if i e u L -i(j),Si(j) = o 

| ^(j)A 11 + (i -^(i))^ 1 if < * Ut-id) 
DiPn + (i - Dt) pf 1 if i = i*(j) 

(23) 

where P/ 1 and P? 1 are the latest successfully shared P/ 1 and 
P? 1 between the secondary transmitter-receiver pair. Finally, 
+ 1) is used to update Whittle's index Wi(j + 1) of each 
channel as detailed in [7]. 

In the case of time-independent channel states, i.e., P/ 1 = 
Pj 01 , the problem reduces to the multi-armed bandit scenario 
considered in [8]. The difference, here, is the lack of the 
dedicated control channel, between the cognitive transmitter 
and receiver, as assumed in [8]. Assuming L = 1, the 
following strategy, which is applied as soon as the initial syn- 
chronization is established, avoids this drawback by ensuring 
synchronization using the ACK feedback over the same data 
channel. 

1) Decision: At the beginning of any time slot j, the 
secondary transmitter and receiver decide to access the 
channel 



2lnj 



arg max ^(j)^], where = 

i— l,--- ,iV 

Xi(j) is the number of time slots 
where successful communication occurs on channel i, 
and Yi(j) is the number of time slots where channel i 
is chosen to sense and access [8]. 
2) Sensing: The secondary transmitter senses channel i* (j). 



3) Access: If §i*(j) = 1, the transmitter sends its data 
packet to the receiver. If the receiver successfully re- 
ceives a packet, it sends an ACK back to the transmitter. 

4) The transmitter and receiver update the following: 

Y i (j + l)=Y i (j) + l, if i(j)=i*(j) 

Xi(j + 1) = Xi(j) + 1, i£Kj.(j) = = 



7i(i + 1) 



YiG+l) 



+ 



2lnj 
YiU+1) 



4 Un-Slotted Primary Network 

In this section, we consider the case of continuous probability 
distributions for the free and busy periods. Whenever the 
channel enters the busy or free state, the time until the next 
state transition is governed by the continuous p.d.f. f T o (y) or 
f T i(x); respectively. Without loss of generality, we will use 
the following exponentially distributed busy/free periods for 
each channel as an illustrative example: 

f n (x) = \ Tl e~ X n x (24) 



fr?(y) = ^T«e 
The channel utilization m in this case is given by: 

E\T?] 



E[T?]+E[Tl] 

Xrpl 

(Xj'i + Ayp ) 



(25) 

(26) 
(27) 



We also assume that the SU is equipped with a single 
antenna that can be used for either sensing or transmission. 

4.1 Multiple Channel Access 

Here, the SU can transmit on any combination of the N 
primary channels simultaneously. However, the SU can sense 
only one channel at a time. Thus in order to sense any channel, 
the transmission taking place on any other channels is paused 
till the end of the sensing event. The goal is to find the 
optimal access strategy that maximizes the throughput for SU 
while satisfying the PU intereference/outage constraints for 
each channel. 

Since the SU depends only on sensing a channel at specific 
times to identify the channel's state, it cannot track the exact 
state transition of each channel. Hence, the free portion of 
time between the actual state transition from busy to free 
until the SU discovers this transition cannot be utilized. In 
addition, some free periods may remain undiscovered at all 
if sensing is infrequent. These Unexplored Opportunities are 
quantified by T/ 7 , which is defined as the average fraction 
of time during which channel f s vacancy is not discovered 
by the SU [3]. On the other hand, the transition of primary 
activity from free to busy on a channel utilized by the SU 
causes interference to the primary and secondary receivers 
until the SU realizes this transition. This Interference Ratio is 
quantified by T/, which is defined as the average fraction of 
time at which channel i is at the busy state but interrupted by 
SU transmission. Finally, we note that blindly increasing the 
sensing frequency to reduce interference and discover more 
opportunities is not desirable because the SU must suspend 
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the use of the discovered channel(s) when it senses other 
channels. This is due to the assumption that data transmission 
and sensing cannot take place at the same time with one 
antenna. Thus the Sensing Overhead Tf is defined as the 
average fraction of time during which channel i discovered 
opportunities are interrupted due to the need for sensing any 
of the N channels [3]. This trade-off will be captured in the 
construction of our objective function which is used to find the 
optimal sensing frequencies/periods. The proposed algorithm 
relies on the novel idea of using two sensing periods for each 
channel: free sensing period Tf if S{(t) = 1, and busy sensing 
period Tf if Si(t) = 0. Therefore, our optimization task is 
to identify the optimal sensing periods Tf*, Tf* for each 
channel, that maximize the total throughput for the SU on the 
N channels while satisfying the PU interference constraint on 
each channel. 

The channel as seen by the SU can be modeled by a two 
state (free/busy) Markov chain, where the transition prob- 
abilities from the free or busy state to the free state are: 

Pf(t) = P(Si(t s + t) = l\Si{t s ) = 1) and P 01 (t) = 
P(Si(t s +t) = l\Si(t s ) = 0), t s is the most recent sensing 
time. For exponentially distributed busy/free periods, P/ 1 (t) 
and -Pj 01 (i) are given by [3]: 



all TV channels, and is given by: 



P?\t) = (1-Ui)+ Ui e 
PP-(t) = (1 - O - (1 - Ui )e 



— (A T i +A T p )t 



— (A T i +A T p )i 



(28) 
(29) 



The ratio of the average number of times the channel is sensed 
free to the total number of times the channel is sensed can be 
obtained from the steady state probability that the Markov 
chain is in the free state: 



pSS _ 



P?HTf) 



l-pn(Tf)+P^(Tf) 



(30) 



In case of perfect sensing (i.e., Pfa = and Pmd = ), 
Pf represents the probability that the SU senses channel i 
with sensing-dependent periods Tf and Tf, and finds it free. 
In the presence of sensing errors, the probability of finding 
channel i free is: 

(1 - Pfa) ■ Pf S + Paid ■ (1 - Pf s ) 

Note that the average time between sensing events on channel 
i is given by: 

fM = Pf s [(l-P FA )Tf + P FA Tf] 

+ (1 - if 5 ) [P MD Tf + (1 - P MD )Tf] (31) 

We define the Secondary Utilization Tf u (Tf,Tf) as the 
expected fraction of time during which channel i is sensed or 
utilized by the SU, 

Tf u (Tf, Tf) = [(1 - P FA )Pf s + P MD (1 - Pf s )] ^ 

Mi 

(32) 

The total SU un-interrupted transmission time is equivalent 
to the expected throughput that can be achieved by the SU on 



A' 



1=1 

N 

= "£[(l- Ui )-Tf(Tf,Tf)-T°(Tf,Tf)] (33) 



Figure [2] illustrates the sensing-dependent periods per chan- 
nel, the interference ratio, the unexplored opportunities, the 
sensing overhead, the secondary utilization, and the secondary 
achieved throughput for a two primary channels model. 

Now, in order to find expressions for the expected un- 
explored opportunities and interference ratio we need first 
to find expressions for 6}(t) and 5®(t), which are defined 
as the expected time in which a channel is free during the 
time between t s and t s + t provided that Si(t s ) = 1 or 
Si(t s ) = 0; respectively. Based on the theory of alternating 
renewal processes, the remaining time x for a channel to be 
in the same state from any sampling time t s can be shown to 

l-F T i/»W 

have the p.d.f. — g jyi/Qj — [15], where F T i/o(x) is the c.d.f. 
of the free or busy period. Therefore, it can be easily shown 
that: 

f°°l-F T i(x) 
$i (t) = t J ^pn'ij — dx 

<■* 1 - F T i (x) 

\ (x + 5°(t-x))dx (34) 
o H->-i\ 



1 - F T o (y) 

ff [T P] s l (* ~ y) d v 



(35) 



where 5}{t) and 5f(t) are the same as $}{t) and $i(t) if the 
change in state happens exactly at t s . That is, 

pOO 

Sl{t) = t / f n (x)dx 



~5°(t) 



+ / f n (x)(x + S?(t-x))dx (36) 



(37) 



fT?(y)6i(t-y)dy 



Using Laplace transform, 5} (t) and 8® (t) for exponentially 
distributed busy/free periods can be obtained as: (see Appendix 
A for a complete derivation) 




^— (\ T o+\ T i)t ^ 

(AyO + Xrpl ) 

-(A T p+A T i)£ ^\ 



(38) 



(39) 



(X F + Xrpl ) I 

The unexplored opportunities Tf can now be obtained as: 

Tf{Tf,Tf) = 

(l^P MD ).(l-PfSy^Ml^ 

+ P FA ■ Pf S ■ ( ffi^ (40) 
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Fig. 2. For a 2 channel primary network, this figure illustrates the periodic sensing using 2 periods per channel, the 
interference ratio, the unexplored opportunities, the sensing overhead, the secondary utilization, and the secondary 
achieved throughput on both channels. 



Similarly, T? is given by: 

T/(7f,lf) = 

(1 - P FA ) ■ Pf 



ss (Tf-6}(Tt 



Paid ■ (1 - if S ) ■ ( ) (41) 



Finally, the sensing overhead Tf is given by: 



N 



if (if , Tf) = (Tf u -Tl)J2[-) (42) 

It is worth noting that the first term (Tf u — T[) represents 
the average fraction of time channel i is utilized by the SU 
without interference from the PU. This is the useful time 
that is interrupted by the need for sensing. The second term 
J2f=i (jt) re P resents tne aggregate sensing overhead given 
by the ratio of the sensing time to the average sensing sensing 
period. 

In summary, given a maximum interference constraint per 
primary channel, Tl max , our optimization problem can be 
expressed as follows: 



Find: Tf*,T q B * 



N 



that maximize: £ T? u (Tf, if) - T?(Tf, Tf) 
i=i 

-7f(lf,7f) 
subject to: if (if , Tf) < if ,i = l,...,N 



This is equivalent to minimizing: 

N 

J2[TF(Tf,Tf)+T°(Tf,Tf)] 

i=l 

subject to the same interference constraint. Tf* and Tf* can 
be obtained numerically (as demonstrated by our numerical 
results in Section 0. 

4.2 Single Channel Access 

Here, we assume that the SU can transmit on only one single 
channel. Our goals are 1) to find the optimal transmission 
period upon accessing any channel in order to satisfy an 
interference/outage constraint on the primary network and 2) 
to find the optimal sequence of primary channels to be sensed 
to minimize the average delay in finding a free channel. 

When the SU finds a channel busy, it switches between 
channels and senses them until a free channel is found. Upon 
finding a vacant channel, the SU transmits for a period of 
Tf. In order to obtain the interference imposed on the primary 
transmission when the SU utilizes the channel for Tf, we note 
that this case is equivalent to setting Tf = in the expressions 
derived in the previous section. Thus, the interference T/(Tf ) 
is given by: 



Ti(Tf) = (l-P FA ) 



P: 



AID ' 



Tf-S}(Tf) 

rpF 

Tf-jpf)f 
TF 



(43) 
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Assuming error-free sensing, the interference T?(Tf) for 
exponentially distributed busy/free periods becomes: 



i 



Tf-{X T? +X T} ) 



(44) 



Let T Imax be the maximum fraction of outage/interference 
that the primary users can tolerate on all primary channels. 
Since only one channel is accessed at a given time, satisfying 
the interference constraint on each single channel is sufficient 
for ensuring that the total interference constraint is satisfied. 
Hence, assuming that the SU sensed channel i to be free at 
time t (i.e., Si(t) = 1), the sensing period Tf for each channel 
must satisfy the constraint: T?(Tf) < T Imax . In order to 
maximize the SU throughput, the optimal sensing period for 
each channel Tf * is Tf which satisfies: (T?(Tf) = T Imax ). 

Now, we focus on the case when a channel is sensed to 
be busy. We need to find the optimal sequence of channels 
to sense in order to find a free channel as soon as possible, 
thereby minimizing the average delay in finding free channels. 
It is shown in [10] that in order to minimize the average 
delay in finding a free channel, assuming all channels have the 
same capacity, the SU should attempt to access the channels 
in descending order of the channel index 7.;, where: 



li 



if 
if 



Si(t s ) 
Si(t.) 



In brief, the proposed strategy works as follows: 

1) Sense the N channels in descending order of ji until a 
free channel is found. 

2) Access the free channel i for the calculated Tf*. 

3) When Tf* ends, recalculate 7$ for all channels, then 
repeat the previous steps. 

In order to synchronize the SU transmitter and receiver to 
the same channel while hopping, in the presence of sensing 
errors or in the case of spatially varying spectrum opportuni- 
ties, we propose using Request To Send (RTS) and Clear To 
Send (CTS) packets. We assume that the packet duration of 
RTS and CTS is negligible compared to Tf for all channels. If 
the transmitter senses a channel free, it will sends RTS. If the 
receiver gets the RTS, it replies with CTS, then transmission 
proceeds for Tf. If the receiver does not receive the RTS, it 
waits for the duration of CTS then switch to the next channel 
and when the transmitter does not receive CTS it also switches 
to the next channel. If the transmitter senses a channel busy, 
it waits for the duration of RTS, then switches to the next 
channel to sense. When the receiver does not receive the RTS, 
it shall switch to next channel. Thus synchronization between 
the secondary transmitter and receiver is always sustained. 
Note that in the case of multiple channel access, the secondary 
receiver is assumed to decode all primary channels, thus, no 
synchronization is needed. 

5 Numerical Results 

5.1 Slotted Primary Networks 

In this subsection we present simulation results for the two 
scenarios discussed earlier in Section [3] Throughout this 
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Fig. 3. Throughput comparison between: the upper 
bound from equation (TO), the proposed blind strategy for 
L = N, the Whittle index strategy for L = 1, the greedy 
strategy for L = 1, and the maximum achievable offline 
bound. 




Fig. 4. Throughput comparison between the proposed 
strategy for (L = N) with and without known transition 
probabilities. 



subsection, we assume that the number of primary channels 
N = 5, each with bandwidth B t = 1. The spectrum usage 
statistics of the primary network were assumed to remain 
unchanged for a block of T = 10 4 time slots for Figures |5] IU 
and [5] and for a block of T = 10 5 time slots for Figure [6] 
The transition probabilities for each channel, i.e., f 11 and 
Pj 01 , were generated uniformly between 0.1 and 0.9. The 
plotted results are the average over 1000 simulation runs. The 
discount factor used to obtain Whittle's index is 0.9999. In all 
our simulations, perfect sensing is assumed and the average 
throughput per time slot is plotted. 

Figure [3] reports the throughput comparison between the 
different cognitive MAC strategies, all with prior knowledge 
about the channels transition probabilities. First, it is noted 
that if the secondary transmitter can sense all N channels 
neglecting the need for synchronization with the receiver, the 
probability of always finding a free channel 1 for large 
N. However, since the strategies we consider sustain the 
secondary pair synchronization, a loss of throughput occurs 
as a price for synchronization. The topmost curve in Figure [3] 
gives the throughput obtained using equation (TTOb assuming 
the delayed side information of all the primary channels' states 
is known to the secondary pair. The proposed strategy for the 
L = N case is shown to achieve a throughput very close 
to the upper bound. Unsurprisingly, there is slight throughput 
gain offered by the full sensing capability as compared to the 
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Fig. 5. Throughput comparison for the blind cognitive 
MAC protocol (with and without the prior knowledge that 
Pi 1 = P? 1 ) anc ' tne genie-aided scenario. 
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Fig. 6. Throughput comparison between the proposed 
blind strategy for (L = 1), when LP = 20 and LP = 200, 
and the genie-aided upper bound. 



L = 1 scenario. The difference, however, is not significant 
due to the constraint of maintaining synchronization which 
dictates choosing a channel to access followed by sensing. It 
is also seen from Figure [3] that the proposed strategies achieve 
a higher throughput than the protocol proposed in [4] where 
the primary transmitter and receiver are assumed to access the 
channels in a predetermined sequence, agreed upon a-priori, 
in which the channel with highest steady state probability of 
being free is always chosen for access. 

Figure [4] illustrates the convergence of the throughput of 
the proposed blind strategy for L = N to the informed 
case with prior knowledge of the transition probabilities as 
T grows. In Figure [5] we assume that P} 1 = Pf 1 for all 
channels. It is shown that even if the secondary users are 
unaware of this fact, and apply the proposed universal strategy, 
the achievable throughput converges asymptotically to the 
achievable performance when the fact that P^ 1 = Pf 1 is 
known a-priori, albeit at the expense of a longer learning 
phase. Interestingly, both strategies are shown to converge 
asymptotically to the genie-aided upper bound (when the 
transition probabilities are known). It is noted in Figures|4]and 
[5] that the proposed blind strategies converge to the strategy 
with known transition probabilities in the range between 10 2 
and 10 3 time slots. 

Finally, Figure [6] demonstrates the tradeoff between the 
learning time overhead in the blind strategy of Section 13.21 
and the final achievable throughput at the end of the T slots 
assuming L = 1. Clearly, this figure supports the intuitive 
conclusion that for large T blocks, one should use a longer 
learning phase in order to maximize the steady state achievable 
throughput. 

5.2 Un-slotted primary channels structure 

In this subsection we present simulation results for the multiple 
channel access scenario discussed in Section |4] In Figures [7j 
and [8] we compare the performance of the proposed strategy 
which adopts the sensing-dependent periods Tf and T B , 
and the strategy proposed in [3] which uses a single sens- 
ing period for each channel. In the simulations, we assume 
N = 5 primary channels with exponentially distributed 
busy/free periods, where A T i = [0.2; 0.17; 0.15; 0.13; 0.11] 
and X T o — [1; 0.9; 0.8; 0.7; 0.6]. Perfect sensing is assumed 



and the channel sensing duration is assumed to be T s = 0.01. 
The plotted results are the average over 100 simulation runs 
and the average throughput per time unit is plotted. The total 
available opportunities in the primary spectrum (upper bound 
on SU throughput) for the given values of Xt 1 and Ay° are: 

— u i)^j = 4.205 . Instead of the assumption that the 

SU will immediately detect returning PUs and evacuate the 
channel, which was used in [3], we impose an interference 
constraint for each primary channel. 

In Figure [7] the interference constraint for 
each channel T[ max = 0.25 m,. Under this 
assumption, our optimization method results in: 
T F * = [0.6133; 0.6800; 0.7637; 0.8714; 1.0148] and 
T B * = [0.3001; 0.3155; 0.3338; 0.3561; 0.3839]. Using 
these values, the expected rate for the SU is given by 
R = 3.8068. The optimization for the strategy proposed in 
[3] results in the single sensing period per channel Tp*, 
where: Tp* = [0.6345; 0.7032; 0.7908; 0.9034; 1.0533] 
and an expected rate of R = 3.7531. In Figure [8] 
we set a more relaxed interference constraint for each 
channel T^ max = 0.75 it,. Our optimization method 
results in: T F * = [3.8847; 4.3127; 4.8462; 5.5318; 6.4457], 
T B * = [0.2793; 0.2950; 0.3135; 0.3359; 0.3637], and 
R = 4.1085. The optimization for the strategy proposed in 
[3] results in: Tp* = [1.0444; 1.1035; 1.1403; 1.1886; 1.2532], 
and R = 3.7731. Overall, we can see from the two figures 
that the throughput of the proposed strategy outperforms 
that of the strategy proposed in [3] at different interference 
constraints. However, one can observe the following trend: 
As the interference constraint becomes more strict (i.e., 
rpimax ^ more frequent sensing is required, resulting 
in a decreased throughput and a reduced advantage of the 
proposed strategy. 

6 Conclusions 

Assuming a slotted structure for the primary network, we 
proposed blind cognitive MAC protocols that do not require 
any prior knowledge about the statistics of the primary traffic. 
Our work differentiated between two distinct scenarios, based 
on the complexity of the cognitive transmitter. In the first, 
the full sensing capability of the secondary transmitter was 
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Fig. 7. Performance comparison between the proposed 
multi-channel access strategy and the strategy proposed 
in [3] for an interference constraint T? max = 0.25 
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Fig. 8. Performance comparison between the proposed 
multi-channel access strategy and the strategy proposed 
in [3] for an interference constraint T? max = 0.75 u. 



fully utilized to learn the statistics of the primary traffic 
while ensuring perfect synchronization between the secondary 
transmitter and receiver in the absence of a dedicated control 
channel. The second scenario focuses on a low-complexity 
cognitive transmitter capable of sensing only a subset of the 
primary channels at the beginning of each time slot. For 
this case, we proposed an augmented Whittle index MAC 
protocol that allows for an initial learning phase to estimate the 
transition probabilities of the primary traffic. Our numerical 
results demonstrate the convergence of the blind protocols 
performance to that of the genie-aided scenario where the 
primary traffic statistic are known a-priori by the secondary 
transmitter and receiver. For un-slotted primary networks, in 
order to maximize the secondary throughput achieved when 
being capable of transmitting on all primary channels simul- 
taneously, we proposed a novel cognitive MAC protocol that 
utilize two sensing periods for each channel depending on the 
sensing outcome. Our numerical results show the superiority 
of the proposed protocol as compared to the protocols that 
rely on optimizing a single sensing period for each channel. 
On the other hand, when the SU can transmit on only one 
single channel, we show the optimal transmission period upon 
accessing any channel in order to satisfy an interference/outage 
constraint on the primary network. 



Appendix A 

Derivation of 8}(t) and 8°(t) 

By taking the Laplace transforms for equations (f34b . ( |35| >. (l36l ) 
and 07): 



*?(•) 
SK*) 



E[T°] 

l-F T i( S ) F T i( S )r5°( S ) 



■E[Tl\ 



fTo(s)Sl(s) 

1-/ti(*) , f T l(s)SUs) 



(45) 

(46) 
(47) 

(48) 



where ¥ T .(t) = 1 - F Ti (t). 
Hence, 



F T o(s) 1-/t.i(s) 



s 2 .E[T?Yl-f Tl (s)f T o(s) 



\E[T}} 



1 - F T i(s). 



1 - / T ° (s) 



1 - /t/(s)/tp(s) 



(49) 
50) 



And since for exponential distributions: 



F Ti (f) 
F Ti (a) 



s + X T , 



s + M 



We reach that: 



s+A„ 



1 - 



s+A„ 



1 - 



s+A T i s+A T p 



S 2 S + (A T o + A T i ) 



(51) 



m = 



y=o 



\ — (A^o +A^i )rr , 

A T p • e L i * ax 



x=0 



(A T o + A T i ) J y=0 
= (1 - im) ■ ( t + 



[1 



dy 

(A T p+A T i)y 



dy 



— (A^o+A^i )t 

e * * — 1 



Similarly, 



5}{t) = t-u i - [t + 



^— (A T p+A T i)f ^ 

(Xj'P + Ayi ) 



(52) 



(53) 
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